Technical Note: Data assimilation and autoregression for using near-real-time streamflow observations in long short-term memory networks

Technical note that compares autoregression to data assimilation for deep learning models and rainfall-runoff modeling.

Abstract

Ingesting near-real-time observation data is a critical component of many operational hydrological forecasting systems. In this paper we compare two strategies for ingesting near-real-time streamflow observations into Long Short-Term Memory (LSTM) rainfall-runoff models: autoregression (a forward method) and variational data assimilation. Autoregression is both more accurate and more computationally efficient than data assimilation. Autoregression is sensitive to missing data, however an appropriate (and simple) training strategy mitigates this problem.

Paper

Nearing, G. S., Klotz, D., Sampson, A. K., Kratzert, F., Gauch, M., Frame, J. M., Shalev, G., and Nevo, S.: Technical Note: Data assimilation and autoregression for using near-real-time streamflow observations in long short-term memory networks, Hydrol. Earth Syst. Sci. Discuss. [preprint], https://doi.org/10.5194/hess-2021-515, in review, 2021.

Code

All code to reproduce the results can be found in this Github repository.

Citation

@Article{nearing2021assimilation,
author = {Nearing, G. S. and Klotz, D. and Sampson, A. K. and Kratzert, F. and Gauch, M. and Frame, J. M. and Shalev, G. and Nevo, S.},
title = {Technical Note: Data assimilation and autoregression for using near-real-time streamflow observations in long short-term memory networks},
journal = {Hydrology and Earth System Sciences Discussions},
volume = {2021},
year = {2021},
pages = {1--25},
doi = {10.5194/hess-2021-515}
}